MSS405052
Design an experiment


Application

This unit of competency covers the skills and knowledge required to design experiments. The design of experiments is generally undertaken as part of black-belt six sigma but may also be undertaken independently.

This unit applies to a technical expert who is required to design and implement experiments aimed at making breakthrough improvements in the process. They will work with other members of the process team in doing this.

This unit primarily requires the application of skills associated with problem solving, initiative and enterprise, and planning and organising skills in order to identify, implement and evaluate an experiment. Communication skills associated with gathering, interpreting and documenting information are required.

No licensing, legislative or certification requirements apply to this unit at the time of publication.


Elements and Performance Criteria

Elements describe the essential outcomes.

Performance criteria describe the performance needed to demonstrate achievement of the element.

1

Choose an improvement project

1.1

Review a process/value stream map.

1.2

Identify areas in need of improvement.

1.3

Select a process/value stream area for analysis and improvement.

1.4

Determine the objective of the experiment in consultation with relevant stakeholders.

2

Design the experiment

2.1

Select appropriate factorial design.

2.2

Estimate signal to noise ratio.

2.3

Determine required number of runs and factorial fraction.

2.4

Determine resolution.

2.5

Design a sequential series of experiments.

2.6

Calculate resource requirement for this design.

2.7

Determine whether resource requirements are practical in consultation with relevant stakeholders.

2.8

Modify experiment, if required, to match available resources.

2.9

Determine/develop required metrics.

3

Conduct the experiment

3.1

Conduct first run of experiment.

3.2

Replicate in random order for required number of runs.

3.3

Block out known sources of variation.

3.4

Conduct other experiments in series.

3.5

Record data/have data recorded.

4

Analyse and confirm the experimental results

4.1

Identify aliases/confounding of variables/results.

4.2

Analyse data using statistics pack or similar.

4.3

Interpret analysed data in line with objectives.

4.4

Identify confidence level of analysed data.

4.5

Design experiment to confirm correlations identified.

4.6

Conduct confirming experiment.

4.7

Analyse data from confirming experiment.

4.8

Confirm results (or conduct further experiments).

Evidence of Performance

Evidence required to demonstrate competence in this unit must be relevant to and satisfy the requirements of the elements and performance criteria and include the ability to design one (1) or more experiments and to:

choose an improvement project

design and conduct the experiment

analyse and confirm the results.


Evidence of Knowledge

Must provide evidence that demonstrates sufficient knowledge to interact with relevant personnel and be able to design an experiment, including knowledge of:

charting, such as Pareto charts, main effects plots, scatter plots, interaction plots, contour plots, response surface plots

statistical principles and analysis, such as analysis of means (ANOM), prediction equations, analysis of variance (ANOVA)/one-way ANOVA, desirability function, hit a target, advanced graphical data analysis, multi-variate planning, variation trees and funnelling, hypothesis testing, central limit theorem, statistical analysis roadmap, analysis for means and t-test, correlation and regression

factorial analysis principles and methods, such as multi-variate analysis, Taguchi S/N ratios, 2/3 level factorial, Taguchi L8, 2/4-1 half fraction, Plackett-Burman 8-run, full factorial

acceptance criteria/confidence levels

appropriate statistics packs, which to choose and how to use.


Assessment Conditions

The unit should be assessed holistically and the judgement of competence shall be based on a holistic assessment of the evidence.

The collection of performance evidence is best done from a report and/or folio of evidence drawn from:

a single project which provides sufficient evidence of the requirements of all the elements and performance criteria

multiple smaller projects which together provide sufficient evidence of the requirements of all the elements and performance criteria.

A third-party report, or similar, may be needed to testify to the work done by the individual, particularly when the project has been done as part of a project team.

Assessment should use a real experiment design project for an operational workplace.

Knowledge evidence may be collected concurrently with performance evidence or through an independent process such as workbooks, written assessments or interviews (provided a record is kept).

Assessment processes and techniques must be appropriate to the language, literacy and numeracy requirements of the work being performed and the needs of the candidate.

Conditions for assessment must include access to all tools, equipment, materials and documentation required, including relevant workplace procedures, product and manufacturing specifications associated with this unit.

Foundation skills are integral to competent performance of the unit and should not be assessed separately.

Assessors must satisfy the assessor competency requirements that are in place at the time of the assessment as set by the VET regulator.

The assessor must demonstrate both technical competency and currency.

Technical competence can be demonstrated through:

relevant VET or other qualification/Statement of Attainment AND/OR

relevant workplace experience

Currency can be demonstrated through:

performing the competency being assessed as part of current employment OR

having consulted with an organisation providing relevant environmental monitoring, management or technology services about performing the competency being assessed within the last twelve months.


Foundation Skills

This section describes those required skills (language, literacy and numeracy) that are essential to performance.

Foundation skills essential to performance are explicit in the performance criteria of this unit of competency.


Range Statement

This field allows for different work environments and conditions that may affect performance. Essential operating conditions that may be present (depending on the work situation, needs of the candidate, accessibility of the item, and local industry and regional contexts) are included.

Competitive systems and practices include one or more of:

lean operations

agile operations

preventative and predictive maintenance approaches

statistical process control systems, including six sigma and three sigma

Just in Time (JIT), kanban and other pull-related operations control systems

supply, value, and demand chain monitoring and analysis

5S

continuous improvement (kaizen)

breakthrough improvement (kaizen blitz)

cause/effect diagrams

overall equipment effectiveness (OEE)

takt time

process mapping

problem solving

run charts

standard procedures

current reality tree.

Objective of the experiment includes one or more of:

screen factors to find the critical few

optimise a few critical factors

solve process problems

reduce waste

increase reliability.

Factorial design includes one or more of:

2/3 level factorial

Taguchi L8

2/4-1 half fraction

Plackett-Burman 8-run

full factorial.

Signal-to-noise ratio may be estimated by one or more of:

previous experiment design experience

previous process capability studies

statistical process control data

estimated from other sources.

Resolution includes one or more of:

Resolution III design: A design where main factor effects are confounded with two factor and higher order interactions

Resolution IV design: A design where main effects are confounded with three factor and higher order interactions and all two factor interactions are confounded with two factor interactions and higher order interactions

Resolution V design: A design where main effects are confounded with four factor and higher order interactions and two factor interactions are confounded with three factor interactions and higher order interactions.

Sequential series of experiments includes all of:

a screening design (fractional factorial) to identify the significant factors

a full factorial or response surface design to fully characterise or model the effects

confirmation runs to verify results.

Required metrics include one or more of:

quantitative measures normally associated with the process

other quantitative measures relevant to the experiment

ranking systems for normally qualitative measures, such as defectives.

Statistics pack include one or more of:

minitab

JMP

other specialist statistics packs

spreadsheets, such as Excel, particularly with specific add-ons, such as Sigma XL, Analyse It or other add-ons.


Sectors

Not applicable


Competency Field

Competitive systems and practices